1. Field of the Invention
The present invention relates to a method of separating specific patterns from a pattern space and an apparatus therefor and, more particularly, to a separation method and apparatus for determining each coupling weight combining neural elements in a hierarchical neural network in accordance with a learning pattern.
2. Description of the Related Art
For example, assuming an n-dimensional pattern space, to recognize a character or image, patterns belonging to a specific group are often separated from a plurality of patterns classified into a plurality of groups in the pattern space.
In separation of specific patterns to recognize characters or the like, a neural network has received a great deal of attention for technologically realizing the mechanism of a human cranial nerve system, thereby performing data processing. A neural network is constituted by a plurality of neuron elements, and predetermined learning allows efficient pattern recognition of characters, figures, and the like, efficient speech analysis and synthesis, efficient continuous analog data processing, and efficient parallel processing of a large number of data.
The neural network is constituted by a neuron element network having a plurality of neuron elements for propagating data and a learning controller for controlling learning in the neuron element network. This neuron element network comprises an input layer for receiving data, an output layer for outputting data in response to the input data, and at least one intermediate layer located between the input and output layers. The neuron elements arranged between the respective layers in the neuron element network are coupled to each other with predetermined strengths (coupling weights). Output signals are changed in accordance with differences in the coupling weights.
In a conventional neural network having a hierarchical structure as described above, the coupling weights between the neuron elements are changed, thereby executing "learning" processing.
Learning is performed by analog or binary data (patterns) supplied in correspondence with the number of inputs and outputs of the input and output layers. Assume that input data is supplied as a learning pattern from the input layer and an output signal is obtained from the output layer in response to this input learning pattern. The correct solution to this output signal is generally called a teacher signal. A plurality of learning patterns are processed for correcting the coupling weights of the respective neuron elements to minimize differences between the output signals and the teacher signals, thereby performing learning.
A practical conventional method of correcting coupling weights between the respective neuron elements in a neuron element network to cause an output signal to coincide with a teacher signal is a back-propagation algorithm (to be referred to as a BP algorithm thereinafter).
According to the BP algorithm, the coupling weights between neuron elements of all layers constituting the neural network are corrected to minimize square errors between teacher signals and output values from the output layer. More specifically, errors in the output layer are judged as a cumulative value of errors occurring in the neuron elements of each intermediate layer. The coupling weights are corrected to minimize not only errors from the output layer but also errors in the neuron elements of each intermediate layer. Therefore, all the errors in the neuron elements of the output layer and each intermediate layer are calculated by computational processing.
In this computational processing, an error value of each neuron element of the output layer is given as an initial value, and backward calculations are performed such that an error value of each neuron element of the nth intermediate layer, an error value of each neuron element of the (n-1)th intermediate layer . . . , are sequentially calculated. Correction values for the coupling weights are then calculated using the error values of the respective neuron elements and the corresponding coupling weights.
Learning processing of all learning patterns is repeated by a predetermined number of times or until an error between each teacher signal and the corresponding output signal becomes below a predetermined value, thereby completing learning.
Learning in a hierarchical neural network by the conventional BP algorithm, however, requires complicated computational error processing for calculating correction values. Therefore, the BP algorithm is undesirably time-consuming until learning is completed.
In addition, the BP algorithm may result in local minima trapping.